Method and apparatus for converting frame rate using time shifting and motion compensation

ABSTRACT

A method of and an apparatus for converting a frame rate using time shifting and motion compensation. The frame rate converting method includes calculating an average motion vector magnitude between every two adjacent frames among a predetermined number of frames input in series determining a frame section corresponding to a maximum value among the average motion vector magnitudes calculated every two adjacent frames and generating a new frame by performing motion compensation, to which a predetermined proportion is applied, in the frame section having the maximum motion vector.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of Korean Patent Application No.2003-71431, filed on Oct. 14, 2003 in the Korean Intellectual PropertyOffice, the disclosure of which is incorporated herein in its entiretyby reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image signal converting apparatusand method, and more particularly, to a method of and an apparatus forconverting a frame rate using time shifting and motion compensation.

2. Description of the Related Art

Commonly, a frame, which is one image picture, is made up of minimumpicture elements, each picture element called a pixel. Resolution isdetermined according to the number of pixels included in the frame. Aframe having a maximum resolution of 1,920×1,080 is 1,920 pixels wideand 1,080 pixels long. A frame rate is the number of frames transmittedper second. When an image signal such as a TV signal is transmitted, anappropriate number of frames per second are transmitted on the basis ofthe eyesight characteristics of a human body.

A frame rate of a film image signal produced for showing a movie iscommonly 24 frames per second (fps). However, a frame rate of anNational Television System Committee (NTSC) standard image signal is 30fps. Therefore, the frame rate of the film image signal must be expandedinto 30 fps (60 fields per second) for applying to an NTSC standardformat.

FIG. 1 illustrates a conventional frame rate converting method suitablefor converting a 24 fps digital image signal into a 30 fps digital imagesignal.

Referring to, FIG. 1, each of 24 input frames F1, F2, F3, F4, . . . ,F24 is divided into a top field f1 and a bottom field f2. New frames F1,F2, F3′, F4′, F5 . . . , F30 are generated using the top fields f1 andthe bottom fields f2 of the 24 input frames F1, F2, F3, F4, . . . , F24.That is, the new frame F3′ is generated from the top field f1 of theframe F2 and the bottom field f2 of the frame F3, and the new frame F4′is generated from the top field f1 of the frame F3 and the bottom fieldf2 of the frame F4.

Likewise, if new frames are generated from adjacent frames whereautocorrelation is high, 24 frames are converted into 30 frames.

The conventional frame rate converting method used to convert a framerate using cross type field repetition generates discontinuity in apicture (commonly, called a motion judder effect) due to simple fieldrepetition. For example, if there is an object flying from the right tothe left in a picture, the motion of the object is inconvenientlydisplayed due to discontinuous portions.

SUMMARY OF THE INVENTION

According to an aspect of the present invention, a frame rate convertingmethod suitable to minimize a motion judder effect due to simple framerepetition by converting a frame rate using time shifting and motioncompensation and an apparatus therefor are provided.

According to an aspect of the present invention, there is provided aframe rate converting method including calculating an average motionvector magnitude between every two adjacent frames among a predeterminednumber of input frames, determining a frame section corresponding to amaximum value among the average motion vector magnitudes calculatedbetween every two adjacent frames, and generating a new frame byperforming motion compensation, to which a predetermined proportion isapplied, in the frame section having the maximum motion vector.

According to another aspect of the present invention, there is provideda frame rate converting apparatus including a frame buffer, which storesa predetermined number of frames of an input video stream; a maximummotion vector determining unit, which measures an average motion vectormagnitude between every two consecutive frames stored in the framebuffer and determines a section having a maximum motion vector among theaverage motion vectors; a frame interpolator, which interpolates bygenerating a new frame on the basis of motion compensation of apredetermined proportion in the frame section having the maximum motionvector determined by the maximum motion vector determining unit; and atiming controller, which generates a timing control signal forsequentially re-combining the frame interpolated by the frameinterpolator and the frames stored in the frame buffer with reference tothe maximum motion vector frame section determined by the maximum motionvector determining unit.

Additional aspects and/or advantages of the invention will be set forthin part in the description which follows and, in part, will be obviousfrom the description, or may be learned by practice of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and/or other aspects and advantages of the present inventionwill become more apparent by describing in detail exemplary embodimentsthereof with reference to the attached drawings of which:

FIG. 1 illustrates a conventional frame rate converting method suitablefor converting a 24 fps digital image signal into a 30 fps digital imagesignal;

FIG. 2 is a block diagram of a frame rate converting apparatus accordingto an exemplary embodiment of the present invention;

FIG. 3 is a flowchart of a frame rate converting method according to anexemplary embodiment of the present invention; and

FIGS. 4A through 4C are concept diagrams for illustrating the flowchartof FIG. 3.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Reference will now be made in detail to the embodiments of the presentinvention, examples of which are illustrated in the accompanyingdrawings, wherein like reference numerals refer to the like elementsthroughout. The embodiments are described below to explain the presentinvention by referring to the figures.

FIG. 2 is a block diagram of a frame rate converting apparatus accordingto an exemplary embodiment of the present invention. Referring to FIG.2, a first frame buffer 210 stores a video stream input at a 24 fps ratein 5 frame units a0, a1, a2, a3, and a4.

A maximum motion vector determining unit 220 includes a motion vectorcalculator 222 and a maximum motion section selector 226. The maximummotion vector determining unit 220 calculates an average motion vectormagnitude between every two consecutive frames stored in the first framebuffer 210 and determines a section having a maximum motion vector amongthe average motion vectors. That is, the motion vector calculator 222calculates the average motion vector magnitude between every twoadjacent frames among the 5 frames. The maximum motion section selector226 compares average motion vector magnitudes calculated between everytwo adjacent frames and selects the frame section having the maximummotion vector.

A frame interpolator 230 generates an interpolation frame by applying apredetermined motion compensation proportion between frames receivedfrom the first frame buffer 210 on the basis of the frame section havingthe maximum motion vector determined by the maximum motion vectordetermining unit 220. For example, if the maximum motion vector isbetween the frame a1 and the frame a2, the frame interpolator 230combines a first frame generated by performing motion compensation ofthe frame a1 by half a motion vector in a direction from the frame a1 tothe frame a2 and a second frame generated by performing motioncompensation of the frame a2 by half a motion vector in a direction fromthe frame a2 to the frame a1. At this time, the combined frame, obtainedby combining the first and second frames, is inserted as a new framebetween a frame b1 and a frame b3.

A switch unit 240 sequentially combines the new frame generated by theframe interpolator 230 and the 5 frames input from the first framebuffer 210 according to a timing control signal from a timing controller260.

A second frame buffer 250 stores the six frames combined by the switchunit 240 as b0, b1, b2, b3, b4, and b5 according to the timing controlsignal.

The timing controller 260 generates the timing control signal andprovides the timing control signal to the switch unit 240 so that theframes are sequentially combined with reference to the maximum motionvector section determined by the maximum motion vector determining unit220. Also, the timing controller 260 generates the timing control signalso that the frames b0 through b5 stored in the second frame buffer 250are output at a frame rate of 30 fps. At this time, the timingcontroller 260 performs time shifting so that the frames b0 through b5stored in the second frame buffer 250 are output with a constant period.

FIG. 3 is a flowchart of a frame rate converting method according to anexemplary embodiment of the present invention.

Five frames of a video stream input at a rate of 24 fps are stored asa0, a1, a2, a3, and a4 in operation 310 in the first frame buffer 210.

Average motion vector magnitudes between each two adjacent frames amongthe 5 consecutively input frames are calculated in operation 320 by themaximum motion vector determining unit 220. For example, as shown inFIG. 4 a, MV1 is the average motion vector magnitude between the framea0 and the frame a1, MV2 is the average motion vector magnitude betweenthe frame a1 and the frame a2, MV3 is the average motion vectormagnitude between the frame a2 and the frame a3, and MV4 is the averagemotion vector magnitude between the frame a3 and the frame a4.

When the average motion vector magnitudes of the frame sections are MV1through MV4, a section having a maximum average motion vector magnitude(MMV) is selected using Equation 1 in operation 330.MMV=Max(MV1, MV2, MV3, MV4)  Equation 1

In operation 340, a new interpolation frame is generated by applyingmotion compensation to the frame section in which the average motionvector magnitude is maximum. Referring to FIG. 4B, a frame interpolationprocess is described in detail. If MV2 is the frame section having amaximum value, new frames b0, b1, b3, b4, and b5 are generated by simplyduplicating existing frames a0, a1, a2, a3, and a4. Applying apredetermined motion compensation proportion to the existing frames a1and a2 generates a new frame b2. That is, a first frame which isgenerated by performing motion compensation of the frame a1 by half amotion vector in a direction from the frame a1 to the frame a2, and asecond frame, which is generated by performing motion compensation ofthe frame a2 by half a motion vector in a direction from the frame a2 tothe frame a1 are combined. At this time, the combined frame is insertedas a new frame b2 between the frame b1 and the frame b3.

The intervals of the newly generated frames b0 through b5 are temporallyirregular as shown in FIG. 4C.

Therefore, rearranged frames c0 through c5 are formed by duplicating theframes b0 and b5 as they are and performing time shifting for the otherframes b1 through b4 in operation 350. For example, the intervals aremade regular by delaying or advancing the frames b1 through b4 bypredetermined times according to the timing controller 260 in thedirections marked by arrows as shown in FIG. 4C.

Thus, the video stream input at the 24 fps rate is converted into avideo stream with a 30 fps rate by applying the time shifting and themotion compensation according to an embodiment of the present invention.

This invention can also be embodied as computer readable codes on acomputer readable recording medium. The computer readable recordingmedium is any data storage device that can store data, which can bethereafter read by a computer system. Examples of the computer readablerecording medium include read-only memory (ROM), random-access memory(RAM), CD-ROMs, magnetic tapes, hard disks, floppy disks, flashmemories, optical data storage devices, and carrier waves (such as datatransmission through the Internet). The computer readable recordingmedium can also be distributed over network coupled computer systems sothat the computer readable code is stored and executed in a distributedfashion.

As described above, according to embodiments of the present invention,an artifact due to motion compensation is reduced by using motioncompensation at the minimum, and visual motion judder is reduced bygenerating a new frame by performing motion compensation in a section inwhich a motion vector is maximum.

Although a few embodiments of the present invention have been shown anddescribed, it would be appreciated by those skilled in the art thatchanges may be made in these embodiments without departing from theprinciples and spirit of the invention, the scope of which is defined inthe claims and their equivalents.

1. A frame rate converting method, comprising: calculating an averagemotion vector magnitude between every two adjacent frames among apredetermined number of input frames; determining a frame section withinthe two adjacent frames corresponding to a maximum value among theaverage motion vector magnitudes calculated between every two adjacentframes; and generating a new frame by performing motion compensation, towhich a predetermined proportion is applied, in the frame section havingthe maximum motion vector, wherein generating the new frame comprises:calculating a motion vector between the two adjacent frames of the framesection; and combining a first frame generated by performing motioncompensation of one of the two adjacent frames by half a motion vectorin a direction towards the other of the two adjacent frames and a secondframe generated by performing motion compensation of the other of thetwo adjacent frames by half a motion vector in a direction towards theone of the two adjacent frames.
 2. The method of claim 1, furthercomprising: performing time shifting of at least one of thepredetermined number of frames and the new frame to form regularintervals.
 3. A frame rate converting apparatus, comprising: a framebuffer which stores a predetermined number of frames of an input videostream; a maximum motion vector determining unit which measures anaverage motion vector magnitude between every two consecutive framesstored in the frame buffer and determines a frame section having amaximum motion vector magnitude among the average motion vectormagnitudes; a frame interpolator which interpolates by generating a newframe on the basis of motion compensation of a predetermined proportionin the frame section having the maximum motion vector determined by themaximum motion vector determining unit; and a timing controller whichgenerates a timing control signal for sequentially combining the newframe interpolated by the frame interpolator and the frames stored inthe frame buffer with reference to the maximum motion vector framesection determined by the maximum motion vector determining unit.
 4. Theapparatus of claim 3, wherein the maximum motion vector determining unitcomprises: a motion vector calculator which calculates the averagemotion vector magnitude between every two adjacent frames among thepredetermined number of frames; and a maximum motion section selectorwhich compares the average motion vector magnitudes calculated by themotion vector calculator between every two adjacent frames anddetermines the frame section having the maximum motion vector.
 5. Theapparatus of claim 3, wherein the frame interpolator combines a firstframe generated by performing motion compensation of one of the twoadjacent frames in the frame section by half a motion vector in adirection toward the other of the two adjacent second frames in theframe section and a second frame generated by performing motioncompensation of the other of the two adjacent frames by half a motionvector in a direction toward the one of the two adjacent frames.
 6. Theapparatus of claim 3, further comprising: an output frame buffer whichsequentially stores the combined new frame and the frames stored in theframe buffer according to the timing control signal generated by thetiming controller.
 7. A frame rate converter, comprising: an inputbuffer which stores frames at a first frame rate; a motion vectordeterminer which calculates an average motion vector value betweenadjacent frames in the input buffer; a frame interpolator whichgenerates a new frame based on the adjacent frames having a maximumaverage motion vector value; and a controller which sequentially formsthe frames and places the new frame between the adjacent frames havingthe maximum average motion vector value, wherein the sequentially formedframes and the new frame have a second frame rate, wherein the frameinterpolator generates the new frame by combining one frame of theadjacent frames having the maximum average motion vector value, and theother frame of the adjacent frames having the maximum average motionvector value according to motion compensation by a predeterminedproportion, and the motion compensation comprises compensating the oneframe of the adjacent frames by half a motion vector toward the other ofthe adjacent frames and compensating the other frame of the adjacentframes by the half of the motion vector toward the one of the adjacentframes.
 8. A frame rate converter, comprising: an input buffer whichstores frames at a first frame rate; a motion vector determiner whichcalculates an average motion vector value between adjacent frames in theinput buffer; a frame interpolator which generates a new frame based onthe adjacent frames having a maximum average motion vector value; acontroller which sequentially forms the frames and places the new framebetween the adjacent frames having the maximum average motion vectorvalue, wherein the sequentially formed frames and the new frame have asecond frame rate; an output buffer which stores the sequentially formedframes and the new frame according to the controller; and a switch whichdirects the frames and the new frame from the input buffer and the frameinterpolator, respectively, to the output buffer according to thecontroller, wherein the frames and the new frame are sequentiallyformed.
 9. The converter of claim 8, wherein the controller time shiftsthe sequentially formed frames and the new frame such that an intervalbetween each frame is constant.
 10. The converter of claim 7, whereinthe first frame rate is 24 frames per second and the second frame rateis 30 frames per second.
 11. A method of converting between a firstframe rate and a second frame rate, comprising: receiving frames of avideo stream at a first frame rate; calculating an average motion vectormagnitude between each pair of adjacent frames in the video stream;determining a maximum pair of adjacent frames corresponding to a maximumvalue among the calculated average motion vector magnitudes; andgenerating a new frame by performing motion compensation to each of theframes of the maximum pair of adjacent frames; and forming a new videostream of sequential frames by inserting the new frame between theframes of the maximum pair of adjacent frames, wherein the new videostream has a second frame rate.
 12. The method of claim 11, furthercomprising: shifting the sequential frames of the new video stream suchthat the frames are evenly spaced.
 13. The method of claim 11, whereinthe generating the new frame comprises: compensating one of the maximumpair of the adjacent frames by a motion vector scaled by a predeterminedamount toward the other of the maximum pair of the adjacent frames; andcompensating the other of the maximum pair of the adjacent frames by thescaled motion vector toward the one of the maximum pair of the adjacentframes.
 14. The method of claim 13, wherein the predetermined amount ofthe scaled motion vector is ½.